1. Field of the Invention
Embodiments herein present a system, method, etc. for an efficient super cluster implementation for solving connected problems in a distributed environment.
2. Description of the Related Art
There is a class of connected problems that involve problem sizes that span a single machine—common examples are Finite Element Model problems that involve simulating large physical systems. One solution to solving this class of problem is to partition the problem across multiple machines. When partitioning a connected problem across multiple machines, the boundary (the internal edges) between the problem pieces must be communicated between the physical machines. With a straight-forward solution to the problem partitioning, the boundary (the edges between the problem pieces) would be communicated between adjacent computers for each problem iteration step. In some cases, namely a slow communication link, it would be beneficial to be able to perform several problem iterations without having to communicate the edges. Presently, the prior art lacks an efficient system and method for managing the problem pieces so that multiple problem iterations may be performed without having to incur a communication step.
One main proposed solution is the “Super Cluster”, wherein by thickening the problem piece edge one “layer”, one iteration's worth of edge communication may be eliminated. Typically, the edge is thickened by the number of iterations one wishes to eliminate. The thicker edge is ad hoc and application specific. Despite its advantages, the “Super Cluster” solution works only for specific solutions in a static setting. It does not work as a general solution, nor does it work in an environment where the problems pieces change in size and shape dynamically. Furthermore, the “Thicker edges” creates a new arbitrary boundary in the problem data structure that is not well-formed, which results in internal chaos in the middleware as it tries to resolve issues with the boundaries being changed arbitrarily.
As such, a solution is required to restore order and regularity in the problem pieces. Also, a solution is needed that works for the dynamically changing problem pieces found in next generation grid middleware systems such as OptimalGrid. Furthermore, this more flexible technique can be combined with other techniques (such as Grid Virtual Memory).